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INTRODUCTION 


Work  done  by  Dudani  (Ref.  1)  at  Ohio  State  University  has 
shown  that  aircraft  can  be  reliably  identified  by  their  silhouettes. 
Dudani  used  digitized  optical  images  of  model  aircraft  from  which 
he  extracted  silhouette  boundaries  and  interior  points.  Using 
moment  invariants  based  on  the  silhouette  boundary  and  interior 
points,  Dudani  constructed  14  features  with  which  he  was  able 
to  classify  the  aircraft  with  great  reliability.  Because  of  the 
success  of  this  work  and  its  significance  to  the  field  of  Identi- 
fication, Friend,  Foe,  or  Neutral  (IFFN)  we  have  investigated  its 
application  to  more  realistic  images. 

Because  Dudani  obtained  his  images  in  a laboratory  environment 
using  clean  white  aircraft  models  with  a solid  black  background,  he 
was  able  to  work  with  pure,  binary  images.  This  eliminated  from  his 
study  many  of  the  problems  encountered  in  the  real  world  imaging 
needed  for  implementation  in  the  field.  In  a previous  study  (Ref.  2) 
we  investigated  the  effect  of  blurring  on  Dudani' s binary  images. 

The  work  reported  here  further  extends  Dudani' s work  in  four  ways. 

We  have 

1)  used  real  aircraft  images  with  full  gray  scale 
illumination 

2)  studied  the  effects  of  background  noise  and  clutter 

3)  studied  the  effects  of  sensor- induced  noise 

4)  applied  our  techniques  to  nonoptical  imagery 


SILHOUETTE  EXTRACTION  FROM  GRAY  LEVEL  IMAGES 


With  the  ultimate  goal  being  aircraft  classification  by  sil- 
houette moments,  the  first  step  must  be  to  process  the  image  in 
such  a manner  as  to  allow  the  extraction  of  a representative  air- 
craft silhouette.  If  the  image  shows  a strongly  backlit  aircraft 
against  a uniformly  brighter  background,  or  vice  versa,  the  sil- 
houette can  be  easily  extracted  by  thresholding  the  image  intensity. 
This,  in  effect,  was  the  situation  Dudani  used  to  simplify  his 
silhouette  determination.  This  fortuitous  situation,  however,  is 
rare  in  actual  aircraft  images. 

We  are  faced,  in  general,  with  the  difficult  problem  of  di- 
viding the  digital  image  into  two  regions.  All  points  representing 
the  aircraft  comprise  the  aircraft  silhouette,  while  all  others 
comprise  the  background.  Our  task  then  is  to  find  the  closed 
bounding  curve  that  separates  these  two  regions.  Since  all  points 
interior  to  this  curve  form  the  silhouette  we  call  this  curve  the 
silhouette  boundary.  In  theory,  the  silhouette  boundary  is  an  ab- 
straction that  lies  on  neither  the  aircraft  silhouette  or  the 
background.  In  fact,  we  will  obtain  it  as  a set  of  digitized 
points  each  of  which  must  lie  in  either  one  region  or  the  other. 

For  our  purposes,  then,  we  will  define  the  silhouette  as  all  those 
points  on  or  interior  to  the  silhouette  boundary. 

Determining  the  silhouette  points  is  then  reduced  to  the 
problem  of  determining  its  boundary.  Features  that  discriminate 
between  the  aircraft  region  and  the  background  region  will  be  those 
that  show  a difference  between  the  two  regions.  With  a view  to- 
wards real-time  implementation  with  high  reliability,  we  need  a 
discriminant  that  is  simple  and  consistently  dependable.  Since 
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the  imaging  equipment  and  data  we  had  access  to  had  no  color  capa- 
bility, we  were  limited  to  gray  level  images.  Texture  offers 
little  promise  since  aircraft  as  well  as  sky  and  cloud  backgrounds 
all  tend  to  be  smooth.  This  leaves  us  with  image  intensity  (gray 
level)  as  a region  discriminant.  Indeed,  one  of  the  most  reliable 
and  straightfoirward  boundary  detectors  is  the  intensity  gradient 
and,  in  particular,  its  local  maxima  (Ref.  2). 

] 

I 
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GRADIENT  LOCAL  MAXIMA  METHOD 


Figure  1 shows  an  example  of  a gray  level  image  of  a bright 
circle  on  a dark  background.  Also  shown  are  ideal  and  typical 
measured  cross  sections  of  image  intensity.  Figure  2 shows  the 
magnitude  of  the  intensity  gradient  and  a crude  boundary  comprised 
of  points  at  which  this  gradient  exceeds  a defined  threshold.  A 
much  sharper  boundary  is  shown  on  Fig.  3 where  only  those  points 
corresponding  to  local  maxima  of  the  gradient  magnitude  are  used. 

In  other  words,  those  points  of  maximum  intensity  difference  serve 
as  the  sharpest  discriminant  between  regions.  Note  also  that  the 
standard  gradient  boundary  is  dependent  on  the  threshold  whereas 
the  gradient  local  maxima  boundary  is  constant  as  long  as  the 
threshold  is  below  the  peaks. 

This  simple  gradient  local  maxima  (GLM)  technique  was  applied 
to  the  test  images  in  the  following  manner  (Fig.  4).  In  each  of 
four  directions  (vertical,  horizontal,  up  diagonal  and  down  diagonal), 
a two-point,  absolute  value  of  intensity  difference  is  taken.  If 
any  of  these  four  directional  derivative  magnitudes  is  not  a local 
maximum  in  its  direction  it  is  set  to  zero.  The  GLM  value  at  the 
point  is  then  set  at  the  maximum  of  the  final  four  values.  An  ex- 
ample of  this  procedure  and  a comparison  with  standard  gradient 
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Fig.  1 Example  of  Image  Intensity  Variation 
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(B)  OBJECT  BOUNDARY 


Fig.  2 Boundary  Determined  by  Gradient 


Fig.  3 Boundary  Determined  by  Gradient  Local  Maxima 
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Fig.  4 Gradient  Local  Maxima  Algorithm 


processing  are  shown  in  Figs.  5 through  8.  Figure  5 shows  an 
image  of  an  A~6  aircraft  against  a clean  background.  The  image 
in  Fig.  6 was  obtained  by  digitizing  the  A“6  photograph  using  a 
Spatial  Data  Systems  Computer  Eye  System  which  produced  a 512 
column  by  480  row  image  of  256  gray  levels.  This  image  was 
reduced  (to  save  computation  time  and  facilitate  image  output) 
to  a 128  x 120  point  image  by  replacing  each  4x4  point  square 
by  the  average  of  its  interior  2x2  squares.  The  digitized  image 
is  represented  on  a computer  terminal  printout  with  the  intensity 
at  each  point  represented  by  one  of  ten  printed  characters.  In  the 
intensity  image  (Fig.  6),  the  higher  the  intensity,  the  darker  che 
character.  In  the  gradient  and  GLM  images  (Figs.  7 and  8),  the 
greater  the  derivative  magnitude,  the  darker  the  character.  Note 
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craft  Image 
Sky) 


Fig.  7 Gradient  of  Aircraft  Image 


Fig.  8 Gradient  Local  Maxima  of  Aircraft  Image 


that  t’;e  gradient  image  (maximum  directional  gradient  at  each 
point  without  regard  to  local  maximum  criterion)  in  Fig.  7 is 
somewhat  fuzzy  while  the  GLM  image  in  Fig.  8 is  quite  sharp  even 
though  the  threshold  used  in  Fig.  7 was  higher  than  that  used  in 
Fig.  8.  Had  the  threshold  in  Fig.  7 been  raised  more  to  sharpen 
the  gradient  image,  significant  boundary  points  (below  the  snorkel 
on  the  nose)  would  have  been  lost. 

BOUNDARY  FOLLOWING 

The  GLM  image  in  Fig.  8 gives  all  the  aircraft  silhouette 
boundary  points  but  includes  points  of  interior  detail  and  ex- 
terior noise.  To  isolate  the  boundary  curve  we  employ  a modified 
version  of  Dudani' s boundary  following  algorithm,  explained  in 
detail  in  Ref.  1.  First  we  threshold  the  GLM  image  and  assign  all 
values  above  the  threshold  a value  of  1 and  all  others  a value 
of  0.  Next  we  scan  the  image  from  left  to  right  and  top  to  bottom 
until  a point  of  value  1 is  hit.  We  call  this  point  "x"  and 
save  it  as  the  starting  point.  Referring  to  Fig.  9,  we  now  search 
the  neighbors  of  point  x counterclockwise  starting  at  Neighbor  2. 
When  we  reach  a point  of  value  1 we  mark  it  as  a boundary  point 
and  it  becomes  the  new  point  x.  If  the  new  point  x was  Neigh- 
bor i of  the  previous  boundary  point  we  start  the  counterclock- 
wise neighbor  search  at  its  Neighbor  i - 2 (Module  8).  This  last 
step  is  repeated  until  the  starting  point  is  reached.  Any  boundary 
point  that  is  encountered  twice  (by  returning  along  a thin  point 
such  as  the  tail  antenna  in  Fig.  8)  is  marked  accordingly  for  later 
use  in  the  silhouette  completion  algorithm.  Note  that  if  small 
noise  blotches  are  circled,  a test  can  be  made  on  the  boundary 
length,  the  points  deleted,  and  the  scan  search  for  the  boundary 
start  point  resumed.  The  boundary  isolated  from  the  GLM  image  is 
shown  in  Fig.  10. 
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Fig.  9 Dudani' s Boundary  Follower  Code 


SILHOUETTE  COMPLETION 

The  silhouette  can  be  filled  in  by  scanning  the  boundary  image 
from  left  to  right  on  each  row  and  marking  as  silhouette  points 
all  those  starting  with  odd  boundary  crossings  and  ending  with  even 
crossings.  (Those  boundary  points  marked  twice  in  the  boundary 
isolation  algorithm  are  counted  twice,  as  both  odd  and  even.)  Some 
holes  will  be  left  in  the  silhouette  because  of  the  scan  line  being 
tangent  to  the  boundary,  but  these  can  be  filled  in  by  scanning  in 
a perpendicular  direction,  and  filling  in  between  boundary  and  in- 
terior silhouette  points.  The  procedure  is  repeated  in  alternating 
perpendicular  directions  until  no  new  silhouette  points  are  found. 

T’le  resulting  silhouette  for  the  A-6  is  shown  in  Fig.  11. 

The  boundary  points  shown  in  Fig.  10  and  the  complete  sil- 
houette points  shown  in  Fig.  11  can  now  be  used  to  calculate  Dudani’ s 
moment  invariants  for  the  A-6. 
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Fig.  10  Aircraft  Silhouette  Boundary 


SILHOUETTE  EXTRACTION  FOR  AIRCRAFT  IN  CLUTTERED  BACKGROUND 


The  A-6  image  used  in  the  preceding  section  was  clean  and 
easy  to  work  with  because  the  background  was  clear  and  uncluttered. 
Often  an  aircraft  will  be  viewed  in  a scene  with  a cluttered  back- 
ground consisting  of  clouds  or  ground  terrain.  To  perform  a con- 
trolled simulation  to  develop  processing  techniques  for  such 
images,  the  A-6  image  was  extracted  from  its  background  using  the 
silhouette  previously  determined.  Figure  12  shows  the  isolated 
A"6  (note  that  the  extraction  was  not  perfect  since,  as  noted  be- 
fore, some  of  the  silhouette  boundary  points  belong  to  the  back- 
ground). The  isolated  aircraft  was  superimposed  on  a cloud  back- 
ground (digitized  from  the  photograph  in  Fig.  13)  to  give  the  com- 
posite image  of  an  aircraft  flying  against  a cloud  background 
(Fig.  14). 

At  this  point  the  GLM  boundary  isolation  and  silhouette  ex- 
traction procedures  were  applied  with  the  results  shown  in  Figs.  15 
and  16.  In  these  images,  the  cloud  background  noise  was  so  strong 
that  no  representative  silhouette  could  be  extracted.  Threshold 
manipulation  did  not  help  because  if  the  threshold  were  raised  to 
the  point  of  separating  the  cloud  background,  significant  points 
in  the  aircraft  boundary  were  lost  so  that  it  was  no  longer  closed. 
More  complicated  boundary  following  logic  might  be  developed  to 
edit  out  noise  or  fill  in  gaps  in  the  boundary.  A more  complicated 
follower  was  tried  but  found  inadequate;  it  seems  the  more  compli- 
cated the  logic  of  the  algorithm  the  more  likely  the  follower  is  to 
get  lost.  A more  straightforward  approach  is  to  reduce  the  back- 
ground noise  in  the  original  image  before  trying  to  extract  the 
silhouette. 

A standard  means  of  reducing  image  noise  is  integration  or 
time  averaging.  Just  as  differentiation  tends  to  enhance  noise, 
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Fig.  12  Isolated  A“6  Image 
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Fig.  14  Composite  Image  of  Aircraft  and  Clouds 
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Fig.  15  Gradient  Local  Maxima  for  Aircraft  and  Clouds 


integration  tends  to  suppress  it.  Since  the  ultimate  application 
of  our  silhouette  extraction  techniques  will  be  on  moving  targets, 
we  can  take  advantage  of  the  availability  of  image  information 
over  a period  of  time  and  average  successive  frames  of  the  image. 

The  important  assumptions  we  must  make  here  are  that 

1)  the  aircraft  is  fixed  in  the  image  frame  during 
the  interval  of  time  over  which  we  average 
(i.e.,  the  sensor  is  locked  onto  the  target) 

2)  the  background  changes  to  some  degree  during 
the  interval  (i.e.,  the  aircraft  is  moving 
across  the  background) 

We  can  relax  the  requirement  of  the  first  assumption  somewhat 
if  we  are  willing  to  take  time  to  shift,  scale,  and  rotate  suc- 
cessive frames  for  maximum  correlation  with  the  running  average. 
However,  the  sim,  lest  application  of  this  procedure  would  be  a 
straightforward  time  exposure.  At  a standard  frame  rate  of  30 
frames  per  second,  1/5  second  exposure  would  allow  6 frames 
to  be  averaged.  In  any  event  the  averaging  could  not  be  effectively 
applied  during  any  time  interval  in  which  the  aircraft  significantly 
changed  its  aspect  relative  to  the  sensor. 

To  test  the  effectiveness  of  frame  averaging,  we  composed  a 
series  of  composite  frames  with  the  extracted  A~6  image  superim- 
posed on  the  cloud  image  at  locations  separated  in  the  horizontal 
direction  by  the  minimum  increment,  one  picture  element.  We  then 
had  a simulation  of  successive  frames  of  an  A-6  moving  across  a 
cloud  background,  with  the  A-6  fixed  and  the  clouds  moving  in  the 
frame . 

The  averaging  technique  was  tested  for  averages  of  3 . 6 , 12 , 
20,  and  30  frames.  No  acceptable  boundary  was  attainable  for  the 
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three  frame  average,  but  for  six  frames  or  more  good  boundaries 
and  silhouettes  were  extracted  (Figs.  17-21). 

In  order  to  give  a quantitative  measure  of  effectiveness  the 
silhouette  moment  invariants  were  calculated  for  each  average.  For 
each  case,  the  Euclidean  distance  between  the  moments  of  the 
average- image  silhouette  and  those  of  the  isolated  A-6  served  as 
a metric.  The  results  are  plotted  in  Fig.  22  where  the  "poor 
classification"  and  "good  classification"  regions  were  determined 
from  averages  of  classification  tests  done  in  a previous  study 
(Ref.  3). 

The  important  result  here  is  that  after  just  six  frames 
(equivalent  to  1/5  of  a second)  an  acceptable  silhouette  was 
extracted  from  a very  noisy  background.  In  a situation  with  either 
less  noise  or  greater  aircraft  motion  across  the  background,  even 
fewer  frames  would  be  required. 
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Fig.  17  Six-Frame  Average  of  Aircraft  and  Cloud 


Fig.  18  Gradient  Local  Maxima  of  Six- Frame  Average 


Fig,  20  Gradient  Local  Maxima  of  Twelve-Frame  Average 


Fig.  2L  Gradient  Local  Maxima  of  Twenty-Frame  Average 


THE  EFFECTS  OF  SENSOR- INDUCED  NOISE 


Reference  3 presents  a study  of  blurring  due  to  sensor  reso- 
lution limitations  and  its  effect  on  silhouette  extraction  and 
classification.  To  complement  this  study  we  investigated  the  ef- 
fect of  sensor  jitter  on  the  silhouette  extraction. 

The  effect  of  jitter  was  simulated  by  applying  an  11  x 11 
pixel  window  to  the  clean  image  of  the  A-6  flying  before  a clear 
background.  The  weighting  of  the  window  was  a truncated  gaussian 
distribution  centered  at  the  center  pixel  and  having  a variable 
standard  deviation  which  was  used  as  a metric  to  quantify  the 
jitter.  The  metric  for  quality  of  the  extracted  silhouette  was 
the  Euclidean  distance  between  moments  of  the  jittered  image  and 
those  of  the  unjittered  image.  The  results  are  shown  in  Fig.  23. 
The  moment  distance  leaves  the  good  classification  region  at  a 
jitter  standard  deviation  (SD)  of  about  ,85  and  enters  the  poor 
classification  region  at  an  SD  of  about  1.3.  At  a distance  of 
100  miles,  with  one  pixel  representing  0.5  feet  in  the  A-6  image 
these  SD  values  correspond  to  angular  values  of  1.7  .j.  and  2.6  a 
radians,  respectively.  These  values  were  compared  to  the  average 
results  of  the  blurring  study  in  Ref.  3 and  agreed  within  a factor 
of  2. 

The  combined  effects  of  jitter  and  background  clutter  were 
simulated  by  actually  shifting  each  of  the  six  composite  A-6/cloud 
images  and  averaging.  The  shift  was  determined  by  a random  number 
generator  with  a gaussian  distribution  of  predetermined  SD.  Since 
shifting  could  only  be  accomplished  on  a quantized  level,  however, 
the  simulation  was  somewhat  crude,  A shift  distribution  with  SD 
up  to  0.6  allowed  successful  silhouette  extraction  with  moment 
distances  in  the  good  classification  range. 
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SILHOUETTE  EXTRACTION  FROM  FLIR  IMAGES 


The  feasibility  of  applying  our  silhouette  extraction  tech- 
niques to  other  sensors  was  tested  on  three  FLIR  images  of  an 
F“ 14  taken  from  near- sequential  frames  of  a video  tape  (Figs.  24 
and  25).  To  reduce  noise  (Fig.  26)  the  three  images  were  averaged 
(Fig.  27)  and  the  GLM  technique  applied  to  obtain  the  silhouette 
(Figs.  28  and  29).  Because  the  average  image  had  reasonably 
distinct  intensity  levels  for  the  aircraft  and  background,  the 
"true’’  silhouette  was  obtained  by  thresholding  the  intensity 
(Fig.  30).  The  GIM  derived  and  true  silhouette  moments  agreed 
within  the  good  classification  region. 


33 


35 


' vrHH«l  * ' ' 

gv»w«,' 

!;?«•!»;«***»• 

/>>?»#««*;***  I *•!:*****«*• 

lilt  111  111 •>»< 

)>)^C<««<«*tlll•lllllllll•••ll***■' 

'<)•  ?i.fj:««ii«tllllil  III!  I mil****  **•**■’ ' . 

'’<Meo«4«-llltllllllllt(**<<i*tl<<**'‘ ****''' 

) ?i;U»«ll»lllllll»  ' 

>> ) '0«i;Ml  l••••••l•*»l  1 1»*  * »»*  *^*  »i**  *• » •*  “*<  ■ * ' 

«>)j)?CCI«*llllllll»»l**i**»»»*»»*»^****-^'»*'*'*''*'' 
>)>»>>)20w«llillll»»i«l»»»*»»»»»‘*»*'«‘" 

• I ) ^ •))»)?€** nil  III*  I III 

.,j,,27«illllHIIIIIIIIIIIIIII»l**ri^if#fjM«.' 

, ( , j » » • ''>«*i»ii»i  III  1 1 1 1 * 1 1 1 ••  I • ' ' * * • ' 

,),..,)*i'J1l4llllll»lllllll****ll‘*l»’*'***'l*******'  " 

.,  ?'>»*iiiiiii»iii»»i*‘ii*iii»'**^*'**' 

•••  .>)0<J)'’’l)'9**lll*»l»*»»l********l**“***^*'**'******’**’ 
i .>)))  )»«>'?;-liiiiiiii»ii*»«»i»»i»»»»i*<*-*:^m*»'V“ 
...  . 1 1 ' I . ) > ) » '?«lii  I nil  1 1 1 III  1 1 III  1 1 II II 1'  *^**  * ’ * ' * * * * * 

. ,Mj  .••)Mi74«nnniiinnniiiiiitn*:nni*i>*>*> 

,,)i  ...  >1.,  . j ,i>«nniiiitnnniiiiiitni*;*n«***w 

, ' . ) ?'<*n  111  I III  1 1 III  I M 1 1 1 nn  I III  • • X 

. ) • > , I II  I • 1 1 1 II  I I I I I I I I I 1 1 I ' 

, ; > II  S I I I II  1 I I till  I I I I I I I i I*  i ' 

. > . . > M Oh*  ki  1 1 lint  1 1 1 1 1 1 1 1 i 1 1 1 M < M • 

:ir«llltlllfClllllllltrlMI**> 
j i 4.1  I I I I III  I I I I 1 1 I 11  i I It  I*  • • 

..  ‘h4«  1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 M 1 1 1 n f • • ■ 

. ■ > -'HCiii  I II II 1 1 1 1 1 1 1 1 II I nil*  • • • 
-41 1 1 n 1 1 mil  nil  1 1 n 1 1 III  • 

4:  i 1 1 1 1 1 1 1 1 1 1 1 III  1 1 1 1 1 II I n I 
'^-n  nil  1 III  I n n 1 1 1 III  n n I 
) ) >onn  II  III  null  1 1 1 1 1 1 1 n 1 1 1 i 
.),>?-«iniiiiiiiiiiiiiinnin4 
j 'i*;4'4ii  1 1 1 111!  im*  4 • 1 1 n n It 

^<:iMl:il||||||ll4»tw--ttlil|t> 

' *>4*  nm  1 1 It  t H *•  'ft-pitpt 

.■;4’4  I I I I M 14  -4p|f 

-4:1  I I I I It  '-t  t t t 

- t I I I t It  ' - -• « t 

*-p  III  It  « t , t 

■ t , I I 1 1 . '-t  • 

...  *Nf4  Ifti  '-t 


'-4  t lit 


P I l< 


38 


Fig.  25  Digitized  FLIR  Images  (Sheet  2 of  3) 
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Fig.  25  Digitized  FLIR  Images  (Sheet  3 of  3) 


Gradient  Local  Maxima  of  First  FLIR  Image 
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Fig.  27  Average  of  Three  FLIR  Images 


CONCLUSIONS 


The  digital  techniques  developed  in  this  study  have  been 
shown  to  possess  the  potential  for  extending  Dudani' s moment 
invariant  classification  method  to  real  world  aircraft  images. 
Aircraft  silhouettes  can  be  effectively  extracted  from  full 
gray  scale  images  by  the  gradient  local  maxima  method.  Even 
images  with  considerable  background  noise  and  sensor  induced 
noise  can  be  effectively  processed.  In  addition,  the  techniques 
have  been  demonstrated  successfully  on  FLIR  imagery. 
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